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SOLID STATE IMAGING APPARATUS 

BACKGROUND OF THE INVENTION 
The present invention relates to a solid state imaging apparatus which is 
5 preferably used for a video camera or the like. More specifically, the present 
invention relates to a knee point control as well as a gain control for the solid 
state imaging apparatus. 

Unexamined Japanese Patent publication No. 3-106269 discloses a 
conventional video signal processing apparatus for a video camera which 
10 controls the characteristics of y correction and knee compression for a video 
signal supplied from an image sensor based on a rate of pixels of the image 
sensor which exceed a predetermined level of the dynamic range. 

SUMMARY OF THE INVENTION 

15 An object of the present invention is to provide a solid state imaging 

apparatus capable of improving image quality of a video signal while improving 
the dynamic range performance. 

To accomplish the above and other related objects, the present invention 
provides a solid state imaging apparatus comprising an image pickup means for 

20 alternately outputting two kinds of video signals within a period of field, the two 
kinds of video signals being a long-term exposure signal having a long exposure 
time and a short-term exposure signal having a short exposure time, and a mixing 
means for mixing the long-term exposure signal and the short-term exposure 
signal at a predetermined brightness level, wherein a setting means is provided 

25 for independently setting a gain and a knee point for each of the long-term 
exposure signal and the short-term exposure signal. Mixing the long-term 
exposure signal and the short-term exposure signal makes it possible to enlarge 
the dynamic range. Providing the capability of independently setting the gain and 
the knee point for each of the long-term exposure signal and the short-term 

30 exposure signal makes it possible to improve the gradation of a mixing portion 
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of the mixed signal. 

Furthermore, according to the solid state imaging apparatus of the present 
invention, it is preferable that the setting means for independently setting the gain 
and the knee point for each of the long-term exposure signal and the short-term 
5 exposure signal comprises a long-term/short-term discrimination pulse generator 
which generates a signal discriminating between a period for the long-term 
exposure signal and a period for the short-term exposure signal, a gain setting 
means for setting a gain, and a knee point setting means for setting a knee point. 
A set value for the gain of the gain setting means and a set value for the knee 

10 point of the knee point setting means are respectively selected based on the 
discrimination signal generated from the long-term/short-term discrimination 
pulse generator, thereby independently setting the gain and the knee point for 
each of the long-term exposure signal and the short-term exposure signal. This 
arrangement makes it possible to arbitrarily setting the gain and the knee point 

15 for each of the long-term exposure time and the short-term exposure time by 
using a single system consisting of a gain adjusting circuit a knee point setting 
circuit. 

Furthermore, according to the present invention, it is preferable that the 
solid state imaging apparatus further comprises a means for detecting an average 

20 brightness value for the short-term exposure signal, and a microcomputer for 
performing an algorithm which is used to calculate the gain and the knee point 
based the detected average brightness value of the short-term exposure signal. 
This arrangement makes it possible to automatically setting the gain and the knee 
point to optimum values in accordance with the condition of the short-term 

25 exposure image. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The above and other objects, features and advantages of the present 
invention will become more apparent from the following detailed description 
30 which is to be read in conjunction with the accompanying drawings, in which: 
Fig. 1 is a diagram showing an arrangement for a knee point control and 
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a gain control of a solid state imaging apparatus in accordance with a preferred 
embodiment of the present invention; 

Fig. 2 is a graph illustrating a mixing operation of exposure signals 
performed in accordance with the knee point control and the gain control of the 
5 preferred embodiment of the present invention; 

Fig. 3 is a timing chart showing a discrimination pulse for discriminating 
between a period for the long-term exposure signal and a period for the short- 
term exposure signal; 

Fig. 4 is a diagram showing an arrangement for a fundamental solid state 
10 imaging apparatus; and 

Fig. 5 is a graph illustrating a mixing operation of exposure signals 
performed the fundamental solid state imaging apparatus shown in Fig. 4. 

DESCRIPTION OF A PREFERRED EMBODIMENT 

15 Hereinafter, knee processing and a gain control of a fundamental solid- 

state image pickup device will be explained. 

Fig. 4 is a diagram showing an arrangement for a fundamental solid state 
imaging apparatus using a double-speed driven image pickup device. 

In Fig. 4, an image pickup device 101 alternately outputs two kinds of 

20 video signals within a period of field at a transmission rate of a horizontal CCD 
which is two times an ordinary value. Of the two kinds of video signals produced 
from the image pickup device 101, one is a long-term exposure signal having a 
long exposure time and the other is a short-term exposure signal having a short 
exposure time. The image pickup device 101 is driven by a pulse signal supplied 

25 from a timing pulse generator 112. A correlation double sampling (CDS) 
processing section 102 converts the video signal produced from the image pickup 
device 101 into a low-frequency signal. An automatic gain control (AGC) section 
103 performs a gain control for the low-frequency signal produced from the CDS 
processing section 102. To expand the dynamic range, a knee processing section 

30 104 compresses a signal whose brightness level is equal to or larger than a 
predetermined value. This is generally referred to as knee processing. The 
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predetermined brightness level is a knee point which is a fixed value. 

An A/D converter 105 receives a knee processed signal from the knee 
processing section 104 and converts this analog signal into a digital signal. A 
time axis converter 106 receives the digital signal from the A/D converter 105 
5 and separates it into a long-term exposure signal and a short-term exposure 
signal. Each of the long-term exposure signal and the short-term exposure signal 
has a standard speed and a same timing. A mixing section 107 produces a mixed 
signal which is substantially the long-term exposure signal at a low-brightness 
region lower than the predetermined brightness level and substantially the short- 

10 term exposure signal at a high-brightness region higher than the predetermined 
brightness level. The low-brightness region and the high-brightness region of the 
mixed signal are smoothly connected at the boundary thereof. A gradation 
correcting section 108 corrects the gradation of the mixed signal. A main signal 
processing signal 111 receives a gradation corrected signal produced from the 

15 gradation correcting section 108. 

An average brightness detecting section 109 detects an average brightness 
value based on the mixed signal. A microcomputer 110 inputs the average 
brightness value detected by the average brightness detecting section 109. The 
microcomputer 110 compares the detected average brightness value with a 

20 predetermined target brightness value. When the detected average brightness 
value is lower than the target brightness value, the microcomputer controls the 
AGC section 103 to increase the gain of AGC section 103, thereby increasing 
the level of a video signal. On the other hand, when the detected average 
brightness value is higher than the target brightness value, the microcomputer 

25 controls the AGC section 103 to decrease the gain of AGC section 103, thereby 
decreasing the level of a video signal. In this manner, to control the level of a 
video signal, the microcomputer 110 calculates an optimum gain to be set in the 
AGC section 103 and controls the gain of the AGC section 103 to the calculated 
value. 

30 According to the above-described fundamental solid state imaging 

apparatus using the double-speed driven image pickup device, it is possible to 
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expand the dynamic range by reducing the exposure time of the short-term 
exposure signal. However, when the exposure time is largely different between 
the long-term exposure signal and the short-term exposure signal, a variation of 
a signal responsive to a change of a light quantity does not match well between 
5 the long-term exposure signal and the short-term exposure signal. Thus, it 
becomes difficult to smoothly mix the long-term exposure signal and the short- 
term exposure signal. The exposure time of the long-term exposure signal is not 
larger than a difference between a vertical scanning time (= approximately 1/60 
sec according to the NTSC system) and a blanking period (=approximately 

10 1/1000 sec). Regarding the exposure time of the short-term exposure signal, the 
blanking period is maximized. To improve the dynamic range performance, it 
may be possible to reduce the exposure time of the short-term exposure signal 
to a level of 1/4000 sec, although it was limited to a level of 1/2000 sec. 
However, this will significantly reduce the smoothness of a mixing portion as 

15 well as deteriorate the gradation. In other words, shortening the exposure time 
of the short-term exposure signal results in deterioration of image quality in 
contradiction to the improvement of the dynamic range performance. 

In view of the foregoing problem, improvement of the dynamic range 
performance essentially requires a system for smoothly mixing exposure signals 

20 without worsening the gradation at a mixing portion regardless of reduced 
exposure time of the short-term exposure signal and also without sacrificing 
improved effects of the dynamic range performance brought by the reduced 
exposure time of the short-term exposure signal. 

Regarding the system for smoothly mixing the exposure signals, it is 

25 possible to reduce the knee point of the long-term exposure signal to a level 
lower than the mixing level only when enlargement of the dynamic range is 
necessary to pick up an image of an object. Next, the gain for the short-term 
exposure signal is increased while maintaining an allowable S/N level. 
Performing this processing prior to the mixing of the exposure signals makes it 

30 possible to equalize a variation of a signal responsive to a change of a light 
quantity to the same value between the long-term exposure signal and the short- 
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term exposure signal. Namely, an inclination of each exposure signal with 
respect to the change of a light quantity can be substantially equalized in the 
vicinity of a mixing portion of the long-term exposure signal and the short-term 
exposure signal. Thus, smooth mixing of the exposure signals can be realized. 
5 However, shortening the exposure time of the short-term exposure signal 

to improve the dynamic range may result in merely increasing the gain for the 
short-term exposure signal and losing the improved effects of the dynamic range 
performance. To avoid this, the knee point of the short-term exposure signal 
should be set to be higher than the mixing level. This is substantially equal to 

1 0 applying the pseudo gamma processing to the short-term exposure signal. In this 
case, as described previously, the knee point of the long-term exposure signal 
must be controlled independently. It is therefore necessary to provide a circuit 
arrangement capable of setting the knee points of the long-term exposure signal 
and the short-term exposure signal to different levels. Regarding the control of 

15 gain, the long-term exposure signal is controlled so that its gain is maximized 
when a low illuminance object is image picked up. However, the short-term 
exposure signal should not be controlled in the same manner. 

More specifically, a picked-up image of an object may include both a low 
illuminance portion and a high illuminance portion. Furthermore, it may be so 

20 dark that the gain for the low illuminance portion must be increased. In such a 
case, the gain for the long-term exposure signal needs to be increased. However, 
if the gain for the short-term exposure signal is increased in the same manner, the 
S/N ratio of a high brightness portion of the picked-up image will be worsened 
unnecessarily. Namely, the gain control for the short-term exposure signal is 

25 characteristically different from that for the long-term exposure signal. Thus, the 
gain control for the short-term exposure signal should be performed primarily for 
realizing the smooth mixing of the exposure signals. 

Accordingly, like setting for the knee point, setting for the gain must be 
performed independently for each of the long-term exposure signal and the short- 

30 term exposure signal. 

However, according to the above-described fundamental solid state 
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imaging apparatus, the long-term exposure signal and the short-term exposure 
signal are time sequentially produced from an image pickup device. This makes 
it impossible to independently set different gains at the AGC section and 
different knee points at the knee processing section for the long-term exposure 
5 signal and the short-term exposure signal. 

To solve the above-described drawbacks, the present invention provides 
a solid state imaging apparatus capable of improving image quality of a mixed 
signal while improving the dynamic range performance by smoothly mixing the 
long-term exposure signal and the short-term exposure signal. 

10 Hereinafter, a preferred embodiment of the present invention will be 

explained. Fig. 1 is a diagram showing an arrangement for a knee point control 
and a gain control of a solid state imaging apparatus in accordance with a 
preferred embodiment of the present invention. 

In Fig. 1, an image pickup device 1 alternately outputs two kinds of video 

15 signals within a period of field at a transmission rate of a horizontal CCD which 
is two times an ordinary value. Of the two kinds of video signals produced from 
the image pickup device 1, one is a long-term exposure signal having a long 
exposure time and the other is a short-term exposure signal having a short 
exposure time. The image pickup device 1 is driven by a pulse signal supplied 

20 from a timing pulse generator 12. A correlation double sampling (CDS) 
processing section 2 converts the video signal produced from the image pickup 
device 1 into a low-frequency signal. An automatic gain control (AGC) section 
3 performs a gain control for the low-frequency signal produced from the CDS 
processing section 2. To expand the dynamic range, a knee processing section 

25 4 compresses a signal whose brightness level is equal to or larger than a 
predetermined value. This is generally referred to as knee processing. 

According to the present invention, a set value for the gain of the 
automatic gain control (AGC) section 3 and a set value for the knee point of the 
knee processing section 4 are switched or changed for each of the long-term 

30 exposure signal and the short-term exposure signal. Thus, the present invention 
makes it possible to independently set the gain and the knee point for each of the 
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long-term exposure signal and the short-term exposure signal. A gain selector 13 
of the automatic gain control (AGC) section 3 sets a gain to be assigned to the 
automatic gain control (AGC) section 3. A knee point selector 14 sets a knee 
point to be assigned to the knee processing section 4. 
5 In this manner, the gain and the knee point can be set independently. A 

long-term/short-term discrimination pulse generator 15, which discriminates a 
period for a long-term exposure signal period from a period for a short-term 
exposure signal and vice versa, is incorporated in the timing pulse generator 12. 
Fig. 3 shows a discrimination pulse produced from the long-term/short-term 

10 discrimination pulse generator 15. This discrimination pulse serves as a 
switching signal for the selectors 13 and 14. 

An A/D converter 5 receives a gain controlled and knee processed signal 
from the knee processing section 4 and converts this analog signal into a digital 
signal. A time axis converter 6 receives the digital signal from the A/D converter 

15 5 and separates it into a long-term exposure signal and a short-term exposure 
signal. Each of the long-term exposure signal and the short-term exposure signal 
has a standard speed and a same timing. A mixing section 7 produces a mixed 
signal which is substantially the long-term exposure signal at a low-brightness 
region lower than the predetermined brightness level and substantially the short- 

20 term exposure signal at a high-brightness region higher than the predetermined 
brightness level. The low-brightness region and the high-brightness region of the 
mixed signal are smoothly connected at the boundary thereof. 

Fig. 5 is a graph illustrating a mixing operation performed in accordance 
with the knee point control of the fundamental solid state imaging apparatus 

25 shown in Fig. 4. 

A mixing level and a mixing width are regulated in such a manner that the 
mixing level is equalized to a 100% level at a final video output. The long-term 
exposure signal Ylong is multiplied with a gain K which starts compression at 
a region exceeding the mixing level, thereby obtaining a signal Kx Ylong. The 

30 short-term exposure signal Yshort is added to an offset value Yoffset. Then, a 
sum signal (Yshort + Yoffset) is multiplied with a gain (1-K) which performs 
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compression at a region below the mixing level, thereby obtaining a signal 
(l-K)x(Yshort + Yoffset). Then, both of the resultant signals are summed to 
obtain a mixed signal Ymix. 

According to the above fundamental solid state imaging apparatus, the 
5 knee point level is a fixed value which is set in the vicinity of 100% level at the 
final video output. As described above, the 100% level is substantially equal to 
the mixing level. 

According to the NTSC system, the exposure time of the long-term 
exposure signal is not larger than a difference between a vertical scanning time 

10 (^approximately 1/60 sec according to the NTSC system) and a blanking period 
(^approximately 1/1000 sec). Regarding the exposure time of the short-term 
exposure signal, the blanking period is maximized. The minimum exposure time 
of the short-term exposure signal is set to 1/2000 sec as a limit for maintaining 
the image quality of a mixing portion. To improve the dynamic range 

15 performance, it may be possible to reduce the exposure time of the short-term 
exposure signal to a level of 1/4000 sec from the fundamental level of 1/2000 
sec. However, this will significantly reduce the smoothness of a mixing portion 
as well as deteriorate the gradation. According to the fundamental example 
which equalizes the mixing level with the knee point level, the relationship 

20 between an incident light quantity and a signal amount is expressed by the graph 
shown in Fig. 5, as the exposure time of the long-term exposure signal is 
approximately 1/60 sec, under the condition the exposure time of the short-term 
exposure signal is set to 1/2000 sec. Namely, a ratio of inclinations of two lines 
is equal to a ratio of two exposure times. 

25 (Inclination of long-term exposure signal) : (Inclination of short-term 

exposure signal) = 1 : 1/32 

Two lines can be mixed smoothly when the inclination of the long-term 
exposure signal is close to the inclination of the short-term exposure signal in the 
vicinity of a mixing point. 

30 On the other hand, to improve the dynamic range performance, it is 

preferable to reduce the exposure time of the short-term exposure signal. When 
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the minimum exposure time is reduced from the fundamental 1/2000 sec level to 
a 1/4000 sec level, the improvement of the dynamic range performance can be 
doubled. The ratio of inclinations of two lines in the vicinity of a mixing point 
becomes as follows. 

5 (Inclination of long-term exposure signal) : (Inclination of short-term 

exposure signal) = 1 : 1/64 

Thus, the image quality of the mixing portion is deteriorated. 
On the other hand, the above-described preferred embodiment of the 
present invention makes it possible to independently set the gain and the knee 
10 point for each of the long-term exposure signal and the short-term exposure 
signal. Accordingly, the knee point for the long-term exposure signal can be 
adjusted to an appropriate value. Independent of the long-term exposure signal, 
settings of the gain and the knee point of the short-term exposure signal can be 
performed adequately as shown in Fig. 2. Then, the two independently adjusted 
15 exposure signals are mixed. 

First, the knee point of the long-term exposure signal is reduced from an 
ordinary 100% level to a 80% level only when an image picked-up object 
requires a dynamic range. A mixing level is maintained at the same 100% or to 
the vicinity thereof. Accordingly, the long-term exposure signal has the mixing 
20 point at a brightness level subjected to compression by the knee processing. The 
smoothness of a mixed signal is doubled compared with the above-described 
fundamental one. 

The short-term exposure signal dedicated average brightness detecting 
section 16 detects an average brightness value of the short-term exposure signal. 

25 The microcomputer 10 inputs the detected average brightness value of the short- 
term exposure signal. Algorithm stored in the microcomputer 10 judges that an 
image picked-up object requires a dynamic range when the average brightness 
value of the short-term exposure signal is large. Thus, the microcomputer 10 
performs a control for reducing the knee point of the long-term exposure signal. 

30 Next, for the short-term exposure signal, its gain is fixed to a doubled 

level. 
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The setting of the gain for the short-term exposure signal is different from 
the setting of the gain for the long-term exposure signal. The setting of the gain 
for the short-term exposure signal is exclusively performed for improving the 
smoothness of the mixed signal. Meanwhile, it is general that the gain for the 
5 long-term exposure signal is controlled to a maximum value when the image 
picked -up object has a low illuminance. The similar gain control should not be 
applied to the short-term exposure signal. More specifically, a picked-up image 
of an object may include both a low illuminance portion and a high illuminance 
portion. Furthermore, it may be so dark that the gain for the low illuminance 

10 portion must be increased. In such a case, the gain for the long-term exposure 
signal needs to be increased. However, if the gain for the short-term exposure 
signal is increased in the same manner, the S/N ratio of a high brightness portion 
of the image will be worsened unnecessarily. Namely, the gain control for the 
short-term exposure signal is characteristically different from that for the long- 

15 term exposure signal. Thus, the gain control for the short-term exposure signal 
should be performed primarily for realizing the smooth mixing of the exposure 
signals. However, solely shortening the exposure time of the short-term exposure 
signal for the purpose of improving the dynamic range may result in losing the 
improved effects of the dynamic range, when the gain for the short-term 

20 exposure signal is fixed to the doubled value. To eliminate this drawback, the 
knee point for the short-term exposure signal is set to a point higher than the 
mixing level. In other words, the knee point for the short-term exposure signal 
is fixed to a level exceeding 100% at the final video output. This is substantially 
equal to applying the polygonal line gamma processing to the short-term 

25 exposure signal, as shown in Fig. 2. Thus, the smoothness of the mixed signal is 
doubled without losing the improved effects of the dynamic range. 

As a result, the embodiment of the present invention makes it possible to 
attain quadruple improvement when the above-described gain and knee point 
control for the short-term exposure signal is combined with the knee point 

30 control for the long-term exposure signal. When the exposure time of the short- 
term exposure signal is 1/4000 sec, the ratio of inclinations of two lines in the 
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vicinity of a mixing point becomes as follows. 

(Inclination of long-term exposure signal) : (Inclination of short-term 
exposure signal) = 1 : 1/16 

Thus, the improvement of the dynamic range performance is doubled 
5 compared with a fundamental one. At the same time, the smoothness of the 
mixed signal at the mixing portion is doubled compared with the above- described 
fundamental one. 

A gradation correcting section 8 corrects the gradation of the mixed signal 
in such a manner that contrast of the gradation is emphasized when the histogram 

10 frequency of an image is large. A main signal processing signal 11 receives a 
gradation corrected signal produced from the gradation correcting section 8. An 
average brightness detecting section 9 detects an average brightness value based 
on the mixed signal. The microcomputer 10 inputs the average brightness value 
detected by the average brightness detecting section 9. The microcomputer 10 

15 compares the detected average brightness value with a predetermined target 
brightness value. When the detected average brightness value is lower than the 
target brightness value, the microcomputer controls the AGC section 3 to 
increase the gain for the long-term exposure signal, thereby increasing the level 
of a video signal. On the other hand, when the detected average brightness value 

20 is higher than the target brightness value, the microcomputer controls the AGC 
section 3 to decrease the gain of AGC section 3, thereby decreasing the level of 
a video signal. In this manner, to control the level of a video signal, the 
microcomputer calculates an optimum gain to be set in the AGC section 3 and 
controls the gain of the AGC section 3 to the calculated value. 

25 As described above, the present invention provides an arrangement for 

independently controlling the gain and the knee point for each of the long-term 
exposure signal and the short-term exposure signal, thereby realizing a smooth 
mixing of the long-term exposure signal and the short-term exposure signal. 
Thus, the present invention improves the dynamic range performance and also 

30 improves the image quality of the mixed signal. 

When a picked-up image of an object comprises both a low illuminance 
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portion and a high illuminance portion, it may be so dark that the gain for the low 
illuminance portion must be increased. In such a case, the gain for the long-term 
exposure signal is increased while the gain for the short-term exposure signal is 
fixed to the same value. Thus, the present invention makes it possible to prevent 
5 the S/N ratio of a high brightness portion of the image from being worsened. 

As described above, the present invention provides a solid state imaging 
apparatus which expands the dynamic range by mixing two kinds of video 
signals which are different from each other in the exposure amount. The present 
invention provides an arrangement for independently controlling the gain and the 

1 0 knee point of each of the long-term exposure signal and the short-term exposure 
signal. With this arrangement, it becomes possible to provides an excellent solid 
state imaging apparatus capable of improving the image quality of a mixed signal 
as a result of smooth mixing of the long-term exposure signal and the short-term 
exposure signal while improving the dynamic range performance. 

15 This invention may be embodied in several forms without departing from 

the spirit of essential characteristics thereof. The present embodiment as 
described is therefore intended to be only illustrative and not restrictive, since the 
scope of the invention is defined by the appended claims rather than by the 
description preceding them. All changes that fall within the metes and bounds of 

20 the claims, or equivalents of such metes and bounds, are therefore intended to be 
embraced by the claims. 
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